Augmented reality (AR) is a technology that superimposes digital information on top of real-world objects. AR applications run on mobile devices, which are equipped with at least one camera and a display. These devices may include tablet computers, head-mounted and other wearable devices, and the like. AR applications track real-world objects in order to align or display digital information alongside or overlaying the objects. Tracking requires an AR application to be capable of identifying an object to be augmented in the camera view. Accordingly, prior knowledge about the object is necessary for it to be recognized. Through preprocessing (e.g. recording of point clouds of real-world objects or manipulation of 3D computer aided drafting (CAD) files), vision algorithms can recognize real-world objects. Tracking and correct alignment might not work properly if the configuration of an object at the time the AR application is in use deviates from the configuration of the same object present during pre-processing.
Mixed Reality (MR) is a technology similar to AR. MR displays digital information in the field of view of a user, mostly using head-mounted devices, such as MICROSOFT HOLOLENS or META 2 devices. In contrast to AR, MR does not track a specific object, but instead tracks the user's environment. MR allows a user to place digital objects at any 3D position in any given real-world room. The 3D position is then fixed in the virtual 3D space. Augmented reality applications require pre-processing in order to superimpose digital information on top of real-world objects. If the configuration of an object that is intended to be tracked changes by the time an AR application attempts tracking, the object might not be recognized. This results in the corresponding digital information not being displayed correctly. Solutions are needed for adapting AR applications to changing configuration objects (e.g., cabinets or computer racks) during runtime using mixed reality (MR) approaches.